Skip to content
This repository was archived by the owner on Sep 10, 2025. It is now read-only.

Update packaging in AOTI path #896

Merged
merged 2 commits into from
Nov 1, 2024
Merged

Update packaging in AOTI path #896

merged 2 commits into from
Nov 1, 2024

Conversation

@angelayi
Copy link
Contributor

@angelayi angelayi commented Jul 11, 2024 •
edited by larryliu0820
Loading

Added a aoti_package path, dependent on pytorch/pytorch#129895. Follow up will be to delete the --output-dso-path.

To export, use the --output-aoti-package-path to specify a file with a .pt2 extension. This will generate an artifact containing all the AOTI generated files

python3 torchchat.py export stories15M --output-aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --device cpu

If we look into the contents of the package, we can see the following:

unzip -l exportedModels/stories15M_artifacts_cpu.pt2

Archive:  exportedModels/stories15M_artifacts_cpu.pt2
  Length      Date    Time    Name
---------  ---------- -----   ----
        1  09-13-2024 08:36   version
        3  09-13-2024 08:36   archive_format
   299452  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.cpp
       26  09-13-2024 08:35   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_metadata.json
   644608  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.o
     1727  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_compile_flags.json
     1738  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_linker_flags.json
173405576  09-13-2024 08:36   data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.so
172941232  09-13-2024 08:36   data/aotinductor/model/cghfy3rmc5pj6omsgyzll5b7btki66najqkyd3544pmitg26ihvj.o
---------                     -------
347294363                     9 files

To run with Python:

python3 torchchat.py generate stories15M --aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --prompt "Hello my name is"

To run with the C++ runner, first build the runner binary:

torchchat/utils/scripts/build_native.sh aoti

To run:

cmake-out/aoti_run exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"

AOTI changes in PyTorch also now allow users to not need to specify --device when generating, as we will save this information during export time as metadata and use it during runtime to determine what device to use. A followup can be to extend the metadata to save model information so that users do not need to specify tokenizer version to use during runtime.

@angelayi angelayi requested review from Jack-Khuu and malfet July 11, 2024 18:08
@pytorch-bot
Copy link

pytorch-bot bot commented Jul 11, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/896

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 32c5807 with merge base f20f5e7 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 11, 2024
@angelayi angelayi changed the title [aoti] Add cpp packaging for aoti [wip][aoti] Add cpp packaging for aoti Jul 11, 2024
@angelayi angelayi force-pushed the angelayi/aoti_python branch 3 times, most recently from b23d414 to 24a1760 Compare July 16, 2024 18:18
malfet
malfet reviewed Jul 16, 2024
View reviewed changes
@angelayi angelayi force-pushed the angelayi/aoti_python branch from 24a1760 to db26ea0 Compare July 16, 2024 20:42
@angelayi angelayi changed the title [wip][aoti] Add cpp packaging for aoti Add cpp packaging for aoti Jul 16, 2024
@angelayi angelayi requested a review from malfet July 16, 2024 21:02
@angelayi angelayi force-pushed the angelayi/aoti_python branch from db26ea0 to 0cf4e99 Compare July 17, 2024 18:37
@angelayi angelayi force-pushed the angelayi/aoti_python branch from 0cf4e99 to 6ec5b35 Compare August 29, 2024 18:07
@angelayi angelayi force-pushed the angelayi/aoti_python branch 3 times, most recently from 94c0079 to 50c6617 Compare September 13, 2024 15:41
malfet
malfet reviewed Sep 13, 2024
View reviewed changes
@angelayi angelayi force-pushed the angelayi/aoti_python branch 2 times, most recently from adb33fb to 0146b38 Compare September 13, 2024 17:01
@angelayi angelayi changed the title Add cpp packaging for aoti Update packaging in AOTI path Sep 13, 2024
@Jack-Khuu
Copy link
Contributor

Weird that the pin bump in #1144 doesn't have the same errors as we see here....

Side question: Is it possible for us to push -l 2 into the dso as metadata as well or does that require looping through pytorch/pytorch? It's an arg when running the binary, but we know AOT what the arg should be

@angelayi angelayi force-pushed the angelayi/aoti_python branch 2 times, most recently from 4311d16 to 937ab9a Compare September 17, 2024 22:30
@angelayi angelayi mentioned this pull request Sep 17, 2024
Merged
@angelayi angelayi force-pushed the angelayi/aoti_python branch 2 times, most recently from f285434 to 2c8b7b5 Compare September 17, 2024 22:53
@angelayi angelayi force-pushed the angelayi/aoti_python branch 2 times, most recently from 925febe to 271f1d1 Compare October 2, 2024 00:03
@angelayi angelayi force-pushed the angelayi/aoti_python branch 2 times, most recently from b9d6ef5 to eae63fe Compare October 2, 2024 18:14
@angelayi angelayi force-pushed the angelayi/aoti_python branch from 9ff2e7d to 93e6727 Compare October 31, 2024 00:28
@angelayi angelayi force-pushed the angelayi/aoti_python branch from 93e6727 to 5cd2862 Compare October 31, 2024 01:10
@byjlw
Copy link
Contributor

byjlw commented Nov 1, 2024

This command fails out of the box. I had to create the directory exportedModels for this to run
the command should create the directories in the specified path if they don't exist already.
python3 torchchat.py export stories15M --output-aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --device cpu

@byjlw
Copy link
Contributor

byjlw commented Nov 1, 2024 •
edited
Loading

command to build the runner fails
torchchat/utils/scripts/build_native.sh aoti

Building aoti native runner...
Defaulting TORCHCHAT_ROOT to /home/warden/source/torchchat/torchchat/utils/scripts/../../.. since it is unset.
~/source/torchchat ~/source/torchchat
Synchronizing submodule url for 'tokenizer/third-party/abseil-cpp'
Synchronizing submodule url for 'tokenizer/third-party/re2'
Synchronizing submodule url for 'tokenizer/third-party/sentencepiece'
~/source/torchchat
-- VERSION: 0.2.1
-- Not Found TCMalloc: TCMALLOC_LIB-NOTFOUND
-- Using ET BUILD DIR: --[et-build]--
-- TORCHCHAT_ROOT="/home/warden/source/torchchat"
-- Looking for excutorch in /home/warden/source/torchchat/et-build/install
-- Could NOT find executorch (missing: executorch_DIR)
-- Caffe2: CUDA detected: 12.0
-- Caffe2: CUDA nvcc is: /usr/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr
-- Caffe2: Header version is: 12.0
-- Could NOT find nvtx3 (missing: nvtx3_dir) 
-- USE_CUDNN is set to 0. Compiling without cuDNN support
-- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
-- USE_CUDSS is set to 0. Compiling without cuDSS support
-- USE_CUFILE is set to 0. Compiling without cuFile support
-- Autodetected CUDA architecture(s):  8.9 8.6
-- Added CUDA NVCC flags for: -gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_86,code=sm_86
-- Configuring done (0.3s)
-- Generating done (0.1s)
-- Build files have been written to: /home/warden/source/torchchat/cmake-out
[1/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_strings_internal.a
[2/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_cordz_functions.a
[3/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/win32_waiter.cc.o
[4/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_strings.a
[5/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/debugging/libabsl_symbolize.a
[6/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_str_format_internal.a
[7/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/crc/libabsl_crc32c.a
[8/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/crc/libabsl_crc_cord_state.a
[9/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_cord_internal.a
[10/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/types/CMakeFiles/bad_optional_access.dir/bad_optional_access.cc.o
[11/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_posix.cc.o
[12/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/types/CMakeFiles/bad_variant_access.dir/bad_variant_access.cc.o
[13/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/types/libabsl_bad_optional_access.a
[14/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_marshalling.a
[15/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/types/libabsl_bad_variant_access.a
[16/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_commandlineflag.a
[17/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/hash/libabsl_hash.a
[18/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_private_handle_accessor.a
[19/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o
[20/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/civil_time.dir/internal/cctz/src/civil_time_detail.cc.o
[21/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_fixed.cc.o
[22/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/zone_info_source.cc.o
[23/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/time/libabsl_civil_time.a
[24/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/perl_groups.cc.o
[25/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_libc.cc.o
[26/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_lookup.cc.o
[27/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/graphcycles_internal.dir/internal/graphcycles.cc.o
[28/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_if.cc.o
[29/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/synchronization/libabsl_graphcycles_internal.a
[30/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_format.cc.o
[31/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/sem_waiter.cc.o
[32/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/pthread_waiter.cc.o
[33/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/kernel_timeout_internal.dir/internal/kernel_timeout.cc.o
[34/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/blocking_counter.cc.o
[35/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/clock.cc.o
[36/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/format.cc.o
[37/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/barrier.cc.o
[38/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/per_thread_sem.cc.o
[39/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/unicode_casefold.cc.o
[40/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/futex_waiter.cc.o
[41/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/notification.cc.o
[42/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/waiter_base.cc.o
[43/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/util/rune.cc.o
[44/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/create_thread_identity.cc.o
[45/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/unicode_groups.cc.o
[46/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/stdcpp_waiter.cc.o
[47/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_impl.cc.o
[48/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o
[49/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/io_win32.cc.o
[50/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/time.cc.o
[51/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/civil_time.cc.o
[52/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/util/strutil.cc.o
[53/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o
[54/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/duration.cc.o
[55/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/mutex.cc.o
[56/127] Building CXX object tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o
FAILED: tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o 
/usr/bin/c++  -I/home/warden/source/torchchat/tokenizer -I/home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src -I/home/warden/source/torchchat/tokenizer/third-party/re2 -I/home/warden/source/torchchat/tokenizer/third-party/abseil-cpp -D_GLIBCXX_USE_CXX11_ABI=0 -MD -MT tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o -MF tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o.d -o tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o -c /home/warden/source/torchchat/tokenizer/tiktoken.cpp
In file included from /home/warden/source/torchchat/tokenizer/tiktoken.cpp:18:
/home/warden/source/torchchat/tokenizer/base64.h:37:11: error: ‘uint32_t’ does not name a type
   37 | constexpr uint32_t DECODE_TABLE[] = {
      |           ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:29:1: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
   28 | #include <string>
  +++ |+#include <cstdint>
   29 | #include <string_view>
/home/warden/source/torchchat/tokenizer/base64.h:57:13: error: variable or field ‘validate’ declared void
   57 | inline void validate(uint32_t v) {
      |             ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:57:22: error: ‘uint32_t’ was not declared in this scope
   57 | inline void validate(uint32_t v) {
      |                      ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:57:22: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:70:3: error: ‘uint32_t’ was not declared in this scope
   70 |   uint32_t val = 0;
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:70:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:72:3: error: ‘uint8_t’ was not declared in this scope
   72 |   uint8_t c = input[0];
      |   ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:72:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:73:12: error: ‘DECODE_TABLE’ was not declared in this scope
   73 |   auto v = DECODE_TABLE[c];
      |            ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:73:25: error: ‘c’ was not declared in this scope
   73 |   auto v = DECODE_TABLE[c];
      |                         ^
/home/warden/source/torchchat/tokenizer/base64.h:74:3: error: ‘validate’ was not declared in this scope
   74 |   validate(v);
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:75:3: error: ‘val’ was not declared in this scope
   75 |   val = v;
      |   ^~~
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode_1_padding(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:105:3: error: ‘uint32_t’ was not declared in this scope
  105 |   uint32_t val = 0;
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:105:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:107:3: error: ‘uint8_t’ was not declared in this scope
  107 |   uint8_t c = input[0];
      |   ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:107:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:108:12: error: ‘DECODE_TABLE’ was not declared in this scope
  108 |   auto v = DECODE_TABLE[c];
      |            ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:108:25: error: ‘c’ was not declared in this scope
  108 |   auto v = DECODE_TABLE[c];
      |                         ^
/home/warden/source/torchchat/tokenizer/base64.h:109:3: error: ‘validate’ was not declared in this scope
  109 |   validate(v);
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:110:3: error: ‘val’ was not declared in this scope
  110 |   val = v;
      |   ^~~
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode_2_padding(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:131:3: error: ‘uint32_t’ was not declared in this scope
  131 |   uint32_t val = 0;
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:131:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:133:3: error: ‘uint8_t’ was not declared in this scope
  133 |   uint8_t c = input[0];
      |   ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:133:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:134:12: error: ‘DECODE_TABLE’ was not declared in this scope
  134 |   auto v = DECODE_TABLE[c];
      |            ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:134:25: error: ‘c’ was not declared in this scope
  134 |   auto v = DECODE_TABLE[c];
      |                         ^
/home/warden/source/torchchat/tokenizer/base64.h:135:3: error: ‘validate’ was not declared in this scope
  135 |   validate(v);
      |   ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:136:3: error: ‘val’ was not declared in this scope
  136 |   val = v;
      |   ^~~
[57/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_info.cc.o
[58/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/parse.cc.o
[59/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/bytestream.cc.o
[60/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/simplify.cc.o
[61/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/statusor.cc.o
[62/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_enum_util.cc.o
[63/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/common.cc.o
[64/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prefilter.cc.o
[65/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/onepass.cc.o
[66/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/status.cc.o
[67/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/tostring.cc.o
[68/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/set.cc.o
[69/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/nfa.cc.o
[70/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arena.cc.o
[71/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/int128.cc.o
[72/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringpiece.cc.o
[73/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/implicit_weak_message.cc.o
[74/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o
[75/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringprintf.cc.o
[76/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arenastring.cc.o
[77/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/coded_stream.cc.o
[78/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/re2.cc.o
[79/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prog.cc.o
[80/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/compile.cc.o
[81/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/parse_context.cc.o
[82/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/message_lite.cc.o
In file included from /usr/include/string.h:548,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/port.h:39,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/macros.h:34,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/common.h:46,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/message_lite.h:45,
                 from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/third_party/protobuf-lite/message_lite.cc:36:
In function ‘void* memcpy(void*, const void*, size_t)’,
    inlined from ‘google::protobuf::uint8* google::protobuf::io::EpsCopyOutputStream::WriteRaw(const void*, int, google::protobuf::uint8*)’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/io/coded_stream.h:699:16,
    inlined from ‘virtual google::protobuf::uint8* google::protobuf::internal::ImplicitWeakMessage::_InternalSerialize(google::protobuf::uint8*, google::protobuf::io::EpsCopyOutputStream*) const’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/implicit_weak_message.h:85:28,
    inlined from ‘bool google::protobuf::MessageLite::SerializePartialToZeroCopyStream(google::protobuf::io::ZeroCopyOutputStream*) const’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/third_party/protobuf-lite/message_lite.cc:419:30:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:33: warning: ‘void* __builtin___memcpy_chk(void*, const void*, long unsigned int, long unsigned int)’ specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
   29 |   return __builtin___memcpy_chk (__dest, __src, __len,
      |          ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
   30 |                                  __glibc_objsize0 (__dest));
      |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~
[83/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prefilter_tree.cc.o
[84/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/dfa.cc.o
[85/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/regexp.cc.o
[86/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_util.cc.o
[87/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_table_driven_lite.cc.o
[88/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/repeated_field.cc.o
[89/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/extension_set.cc.o
ninja: build stopped: subcommand failed.

@angelayi
Copy link
Contributor Author

angelayi commented Nov 1, 2024

the command should create the directories in the specified path if they don't exist already.

That should be fixed with pytorch/pytorch#138919

@larryliu0820
Copy link
Contributor

the command should create the directories in the specified path if they don't exist already.

That should be fixed with pytorch/pytorch#138919

Seems like we need to bump the pin to pytorch as well?

@Jack-Khuu
Copy link
Contributor

https://github.com/pytorch/torchchat/pull/1319/files

There's a bump here that's not actively being worked on, seems like it's not a free bump (+ the infra issue we're seeing here)

@byjlw
Copy link
Contributor

byjlw commented Nov 1, 2024

currently running into this issue when trying to run using the runner.

(.venv) warden@Vikander:~/source/torchchat$ cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"
2024-11-01:14:45:53,463 INFO     [sdpa_with_kv_cache.py:29] Loading custom ops library: /home/warden/source/torchchat/.venv/lib/python3.11/site-packages/executorch/extension/llm/custom_ops/libcustom_ops_aot_lib.so
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to initialize zip archive: file open failed
Aborted (core dumped)

@byjlw
Copy link
Contributor

byjlw commented Nov 1, 2024

currently running into this issue when trying to run using the runner.

(.venv) warden@Vikander:~/source/torchchat$ cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"
2024-11-01:14:45:53,463 INFO     [sdpa_with_kv_cache.py:29] Loading custom ops library: /home/warden/source/torchchat/.venv/lib/python3.11/site-packages/executorch/extension/llm/custom_ops/libcustom_ops_aot_lib.so
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to initialize zip archive: file open failed
Aborted (core dumped)

path issue. Just need to update the instructions in the first comment so it's

cmake-out/aoti_run exportedModels/stories15M_artifacts_cpu.pt2 -z python3 torchchat.py where stories15M /tokenizer.model -l 2 -i "Once upon a time"
and not
cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z python3 torchchat.py where stories15M /tokenizer.model -l 2 -i "Once upon a time"

@larryliu0820 larryliu0820 merged commit 74584d9 into main Nov 1, 2024
52 checks passed
@Jack-Khuu Jack-Khuu mentioned this pull request Nov 5, 2024
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@Jack-Khuu Jack-Khuu Awaiting requested review from Jack-Khuu

2 more reviewers

@malfet malfet malfet left review comments

@larryliu0820 larryliu0820 larryliu0820 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@angelayi @Jack-Khuu @byjlw @larryliu0820 @malfet @facebook-github-bot